IMPORTANT NOTES:

• THIS PR SHOWS HOW TO USE WEAVER TO GENERATE CODE FOR SEMANTIC CONVENTIONS.
• THE GENERATED RUST CODE IS JUST AN EXAMPLE, NOT AN OFFICIAL RUST SEMCONV VERSION.

Guidance to review this PR

The purpose of this PR is to allow various maintainers of API/SDK clients to provide feedback on Weaver in the context of code generation. The generation of Rust code for SemConv is taken as an example.

File structure, under crates/weaver_codegen_test/:

• semconv_registry/: Contains the SemConv registry used in this example.
• templates/registry/rust/: Holds the templates for this PR.
• templates/registry/rust/weaver.xml: Defines the JQ expression for applying each template to the resolved registry.
• generated_code/: Location where the Rust SemConv code is generated.

The following command can be used to reproduce this code generation:

cargo run -- registry generate \
-t crates/weaver_codegen_test/templates/ \
-r crates/weaver_codegen_test/semconv_registry \
rust crates/weaver_codegen_test/generated_code/

To the attention of API/SDK client maintainers,

Weaver uses a Jinja2-compatible engine to render the templates. The following custom filters are available:

• lower_case: Converts a string to lowercase.
• upper_case: Converts a string to UPPERCASE.
• title_case: Converts a string to TitleCase.
• pascal_case: Converts a string to PascalCase.
• camel_case: Converts a string to camelCase.
• snake_case: Converts a string to snake_case.
• screaming_snake_case: Converts a string to SCREAMING_SNAKE_CASE.
• kebab_case: Converts a string to kebab-case.
• screaming_kebab_case: Converts a string to SCREAMING-KEBAB-CASE.
• acronym: Replaces acronyms in the input string with the full name defined in the acronyms section of the weaver.yaml configuration file.
• split_ids: Splits a string by '.' creating a list of nested ids.
• type_mapping: Converts a semantic convention type to a target type (see weaver.yaml section type_mapping).
• comment_with_prefix(prefix): Outputs a multiline comment with the given prefix.
• flatten: Converts a List of Lists into a single list with all elements.
  e.g. [[a,b],[c]] => [a,b,c]
• attribute_sort: Sorts a list of Attributes by requirement level, then name.
• metric_namespace: Converts registry.{namespace}.{other}.{components} to {namespace}.
• attribute_registry_file: Converts registry.{namespace}.{other}.{components} to attributes-registry/{namespace}.md (kebab-case namespace).
• attribute_registry_title: Converts registry.{namespace}.{other}.{components} to {Namespace} (title case the namespace).
• attribute_registry_namespace: Converts metric.{namespace}.{other}.{components} to {namespace}.
• attribute_namespace: Converts {namespace}.{attribute_id} to {namespace}.

Note 1: Most of the standard Jinja2 filters are also supported.

Note 2: Some filters present in the build-tools have not yet been implemented in Weaver. It would be interesting to get feedback from the maintainers to identify which are missing.

The following custom testers are available:

• stable: Tests if an Attribute is stable.
• experimental: Tests if an Attribute is experimental.
• deprecated: Tests if an Attribute is deprecated.

To the attention of the maintainers of opentelemetry-rust

@jtescher, @TommyCpp , @cijothomas , @lalitb , @djc (and any other opentelemetry-rust maintainers/contributors), please review this PR and provide feedback on both the Weaver tool and the Rust code generated from a few Jinja2 templates that I assembled specifically for this exercise.

The generated code adheres to the recommendations defined in this PR by @lmolkova, along with a few other suggestions of my own:

• One Rust file per group namespace.
• Every SemConv experimental attribute is annotated with #[cfg(feature = "semconv_experimental")]
• Every SemConv deprecated attribute is annotated with #[deprecated(note="{{ attribute.deprecated }}")].
• Every SemConv enumeration is translated into a non-exhaustive Rust enum. Each of these enums implement the Display trait and expose a method as_str returning the string value for each variant. A _Custom variant is also added to let the users define their own variant.
• Every attribute of the registry is declared as a typed AttributeKey constant. A method is exposed to generate a standard KeyValue from a properly typed given value.
• For the metrics, this example generates an API similar to the one used by the Python SDK, as well as a type-safe API (more details below).

Below is an example showcasing the creation of metrics using the type-safe API generated by the templates. The goal is to guide our users through the smart completion offered by most IDEs (including documentation), and also to leverage Rust's type system and compiler to guide the user and prevent any omissions, errors, and misuse.

    // ==== A TYPE-SAFE HISTOGRAM API ====
    // Create a u64 http.server.request.duration metric (as defined in the OpenTelemetry HTTP
    // semantic conventions)
    let http_request_duration = HttpServerRequestDuration::<u64>::new(&meter);

    // Records a new data point and provide the required and some optional attributes
    http_request_duration.record(100, &HttpServerRequestDurationReqAttributes {
        http_request_method: HttpRequestMethod::Connect,
        url_scheme: "http".to_owned(),
    }, Some(&HttpServerRequestDurationOptAttributes {
        http_response_status_code: Some(200),
        ..Default::default()
    }));

    // ==== A TYPE-SAFE UP-DOWN-COUNTER API ====
    // Create a f64 http.server.request.duration metric (as defined in the OpenTelemetry HTTP
    // semantic conventions)
    let http_client_active_requests = HttpClientActiveRequests::<f64>::new(&meter);

    // Adds a new data point and provide the required attributes. Optional attributes are not
    // provided in this example.
    http_client_active_requests.add(10.0, &HttpClientActiveRequestsReqAttributes {
        server_address: "10.0.0.1".to_owned(),
        server_port: 8080,
    }, None);

    // ==== A TYPE-SAFE COUNTER API ====
    // Create a f64 system.cpu.time metric (as defined in the OpenTelemetry System semantic
    // conventions)
    let system_cpu_time = SystemCpuTime::<f64>::new(&meter);

    // Adds a new data point and provide some optional attributes.
    // Note: In the method signature, there is no required attribute.
    system_cpu_time.add(10.0, Some(&SystemCpuTimeOptAttributes {
        system_cpu_logical_number: Some(0),
        system_cpu_state: Some(SystemCpuState::Idle)
    }));
    // Adds a new data point with a custom CPU state.
    system_cpu_time.add(20.0, Some(&SystemCpuTimeOptAttributes {
        system_cpu_logical_number: Some(0),
        system_cpu_state: Some(SystemCpuState::_Custom("custom".to_owned()))
    }));

    // ==== A TYPE-SAFE GAUGE API ====
    // Create a i64 system.cpu.utilization metric (as defined in the OpenTelemetry System semantic
    // conventions)
    let system_cpu_utilization = SystemCpuUtilization::<i64>::new(&meter);

    // Adds a new data point with no optional attributes.
    system_cpu_utilization.record(-5, None);
    // Adds a new data point with some optional attributes.
    system_cpu_utilization.record(10, Some(&SystemCpuUtilizationOptAttributes {
        system_cpu_logical_number: Some(0),
        system_cpu_state: Some(SystemCpuState::Idle)
    }));

Note 1: Similarly to other Client SDKs, the individual SemConv attributes and metrics are accessible via the generated code without using the type-safe API (as seen in the Python or Java SDKs). We could offer a configuration mechanism allowing users to choose between permissive (but error-prone) APIs or more rigid, type-safe APIs.

Note 2: A similar approach could be used for Span, ...

Discussion on performance:

• The current experimentation generates code relying on the opentelemetry and opentelemetry-sdk crates. The entirety of the existing pipeline is reused. The type-safe API generated on top of the SDK is relatively light but adds a small overhead.
• It is possible in the long run to completely eliminate the numerous layers of abstraction by directly generating the code that will interface this type-safe API with the opentelemetry-otlp crate (or any other target exporter). With this approach, many optimizations are possible: Direct conversion of Rust structures representing required and optional attributes to corresponding protobuf messages. Aggregation relies on computing a hash combining all key+value attributes sorted by key; it is possible to completely eliminate this sorting as well as half of the overhead of the hash (hashing of values only). Many other optimizations are possible with this type of approach.

Status

• Implement the Display trait for every enum.
• Expose the method as_str for every enum.
• Generate deprecated and experimental attributes.
• Use the first segment in the attribute id instead of the prefix of groups.
• Generate Metrics
• Test code generate (see crate weaver_codegen_test).
• More documentation on JQ syntax.

Work for future PR: Generate type-safe API for spans.